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Appearance Personalty Aflrltxjt* - 

Male Female 

< 

Skin tow: 3/5 

> 

€ 

Hairstyle 5/4 

> 

< 

Hair colour: 3/4 

> 

< 

Eye colours 5/3 

> 

< 

Upper clothing: i/3 



Select Shirt Colour 


< 

Lower eJmhlng 1/3 

> 


Select Pants Colour 


< 

Shoes: 1/2 

> 

L 

Done 



Continue in: 3; 1 2 





University Research aims to be 

generalizable 

Game development aims to 
answer game specific 
questions 



Angus hates Aliens | Team Stendec 
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How to improve? 

What about developers? 
How about play testers? 


1 1 Mill 11 



Your Design 

Optimized at Local Maxima 
Current 


A Better Design 
/ 




Does the designer's idea match the 
player's experience? 

= > the death of a beloved character 





Hands up: 

Who only tests with team-members? 
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Hands up: 

Who also runs tests with friends and family? 




Hands up: 

Who runs planned play tests 
with paid participants? 
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Time & Money 
!Core to your Product 
Return on Investment? 
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Requirement Development Deployment 
Phase Phase Phase 
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2. Logistics of scheduling particip 

3. Physically present participants 

4. Financial resources 
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Crowdsourcing 
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Haifa million workers 
Diverse sample 
Remotely available 




The Worker 

mean age ~ 35 years 
~ 45% female / 55% male 
~ 47% married / 40% single 
~ 60% bachelor degree or above 
~ 60% identify as playing games 



N=400 


■ III 
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Accounts are valuable 


Approval rate, e.g., 90% 

No. of HITs approved, e.g. 20.000 


Master worker - very high ■ 

consistency, across HITs and requester 
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HIT Preview 


Instructions 


We are conducting an academic study about personalizing, adapting, and balancing computer games. You will be asked to fill out 
questionnaires, play a game, and perform a task. Following game and task, you will be asked to complete additional questionnaires that 
ask you questions about your experience. 

This task can only be performed in Fi refox , since it requires Unity. 

Select the link below to complete the study. At the end of the study, you will receive a code to paste into the box below to receive credit 
for taking our survey. 

Make sure to leave this window open as you complete the survey. When you are finished, you will return to this page to paste the 
code into the box. 


Survey link: 

Provide the survey code here: 


http://hci-mturk.usask.ca: 8670 

e.g. 123456 


Submit 





Recorded Play & Interviews 
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Surveys 

- Multi-purpose: 

E.g., Demographics, 
experience 
measures, written 
feedback etc. 



mi 




Logging 
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Logging 
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Physiological Signals 



III II 1 111 



GDC 


GAME DEVELOPERS CONFERENCE EUROPE COLOGNE, GERMANY 15-16 AUGUST 2016 


Awesome! So, no problems? 



Multiplayer 

- Cue workers 

- Larger samples 



mi 




Protect your Intellectual Property?! 


• Difficult for AAA 

• Protection? 

• Visibility > concerns 
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Take away 

Crowdsourcing playtesting: 

• Gives access to playtests independent of studio 

• Allows to test your game early on 

• Saves time and money 
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